This thesis is a study of pattern classification based on information theoretic criteria. Information theoretic criteria are important measures based on entropy and divergence between data distributions. First, the basic concepts of pattern classification with the well known Bayes classification rule as a starting point is discussed. We discuss how the Parzen window estimator may be used to ...